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their approval of the program. The author indicates performance contracting acted 
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PllEFACE 



Thi‘s Report is a product of Rand’s study of performance contracting in educa- 
tion. The study is sponsored by the Assistant Secretary for Planning and Evaluation, 
U.S, Departmsnt of Health, Education and Welfare, under Contract No. HEW-OS- 
70-156. 

Case Studies in Educational Performance Coni "acting comprises six volumes. 
Each is a self-contained study; together they provide a multifaceted view of perform- 
ance contracting. The six volumes are: 

1. R-900/1-HEW, Conclusions and Implications, by P. Carpenter and 
G. R. Hall 

2. R-900/2-HEW, Norfolk, Virginia, by P, Carpenter 

3. R-900/3-HEW, Texarhana, Arkansas and Liberty-Eylav, Texas, by P. 
Carpenter, A. W. Chalfant, and G, R. Hall 

4. R-900/4-HEW, Gaiy, Indiana, by G. R. Hall and M. L. Rapp 

6. R-900/6-HEW, GUroy, CaUfbrnia, by M. L. Rapp and G. R. Hall 

6. R-900/6-HEW, Grand Rapids, Michigan, by G. C. Sumner 

This study is the second of three Rand Reports on the subject. The first Report 
was J. P. Stucker and G. R. Hall, The Performance Contracting Concept in Education, 
The Rand Corporation, R-699/1-HEW, May 1971. The third Report will be a per- 
formance contracting guide intended for use by educational officials. 



SUMMARY 



The Gilroy, California, Unified School District contracted with Westinghouse 
Learning Corporation for the 1970-71 school year, in hcipes of improving the reading 
and mathematics achievement of approximately 100 T itle I students, most of whom 
tested below gi’ade level at the start of the program. The students were mostly 
Spanish-surnamed boys and girls, from grades 2, 3, and 4. 

As to be expected, a number of problems arose during the year that had to be 
resolved for the program to continue. These problems arose largely for two reasons. 
First, WLC did not have a completely developed curricu lum, nor was WLC prepared 
for the very low entering achievement level of some of the students, especially in 
the second grade. Second, problems arose because the stafi’ of the district had not 
been sufficiently involved in making the decision to implement a performance con- 
tract, This was especially true of those teachers in Eliot School who were not directly 
involved in the progi*am, but whose students attended the center for their instruc- 
tion in reading and mathematics. There was a cooperative spirit between the pro- 
gram teachers and the Weatinghouse on-site director which enabiad each to take 
advantage of the other’s expertise in improving the program as it progressed during 
the year. 

Achievement results did not come close to the expectations of either the school 
district or the contractor. Gilroy had hoped for two achievement-years of gain for 
each student in both reading and mathematics during the school year. Instead, the 



average gain was 0.6 achievement-years in reading and 0.8 achievement-years in 
mathematics. 

An unusual feature of the Gilroy program was that some students took both the 
Metropolitan Achievement Test and the Stanford Achievement Test on a pre- and 
post-test basis. This feature provided both a "backup” test and also an alternative 
measure of cognitive growth. There ^^as no significant difference between mean 
achievement gains as measured by the two instruments. 

An analysis of nonacademic results revealed that: (1) students who were inter- 
viewed generally enjoyed the program, (2) there was no observable efFect of the 
program on student attendance, and (3) of the S3 parents who returned a question- 
naire eliciting their opinion of the program, 48 were pleased with it. 

Despite the disappointing achievement, performance contracting acted as an 
agent for positive change. The staff of Eliot School, including the teachers who were 
not involved in the program this year, have submitted a proposal to the Superintend- 
ent to run their own reading resource center at the school next year. They found 
the systematic approach taken by Westinghouse a model around which they could 
organize their efforts for changing pupil behavior. The teachers are more concerned 
today with diagnosing a child’s performance in relation to the skills he needs to 
develop, and then prescribing for him an instructional curriculum that is specific to 
his individual needs. Because there were two instructional aides in the center, and 
because the program was individualized, the teacher’s role changed from that of 
being the traditional imparter of information to acting as a manager of learning 
experiences. 
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I. INTRODUCTION 



This Report analyzes the perforniance contracting experience of the Gilroy 
(California) Unified School District (GUSD) during 1970-71 and its implications for 
other school districts considering performance contracts. Gilroy’s experience is in- 
structive. GUSD serves sizable Mexican-American and migrant worker communi- 
ties and has challenging compensatory education responsibilities. Like many other 
school districts, it is small and has more limited curriculum development resources 
and other support services at its disposal than those enjoyed by larger school dis- 
tricts. The performance contracting program that Gilroy sponsored was a “bare- 
bones” model lacking many of the features of larger programs, such as competitive 
source selections, management support contractors, independent evaluation con- 
tracts, and so forth. The very austerity of the program offers some interesting 
advantages to the researcher, however. Gilroy permits examination of a very 
straightforward approach to performance contracting, as well as the contribution of 
one performance contract to compensatory education in a smaller school district. 

Section II gives some background details about Gilroy. Sectioris III and IV 
describe the program. Section V analyzes the academic achievement outcomes of the 
program, and Section VI discusses other aspects of the program. The final section 
presents some conclusions. , • 





II. GILROY, CALIFORNIA 



Gilroy, California, is in the southern Santa Clara Valley (see Fig. 1). Its popula- 
tion was 11,250 in the April 1970 census. Approximately 34 percent have Spanish 
surnames. U.S. 101 and the Southern Pacific railroad run through the center of 
town. To the east there is a hea^^ concentration of Spanish-surnamed families. To 
the west the city is largely Anglo. The performance contracting program that is the 
subject of this study was located at Eliot School east of the tracks (see Fig. 2). 

The Gilroy area is predominantly agricultural but because of ite proximity to 
San Jose, 35 miles to the north, there is increasing occupational diversification. San 
Jose is the home of a large division of Lockheed, IBM has offices there, and much 
light electronic manufacturing has come to the San Jose area In the wake of these 
corporations. 

Gilroy Unified School District has a student population of approximately 6000, 
from kindergarten through the twelfth grade. Approximately 64 percent of tiie 
students come from families whose annual income is less than $3000. Approxi- 
mately 21 percent of the Spanish-surnamed population are receiving assistance 
under the Aid to Families with Dependent Children program. 

The GUSD is headed by an elected school board that is very active in school 
policy formation. Two board members are Mexican-Americans. The board approved 
the performance contracting program on em experimental basis, but as will be 
discussed later, the initial .impetus for the program came from the school adminis- 
tration. 
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Fig. 1— Location of Gilroy, Cidifornia 






■Locatio'n of Eliot School, Gilroy 



There is a strong teachers union in Gilroy that is an active commentator on 
school district affairs. Despite national union opposition to performance contracting, 
the local union was persuaded to accept the performance contracting program on 
a trial basis. The local PTAs are weak and were not much of a factor in the program, 
GUSD has striven with some success to raise the achievement scores of its 
students, but achievement scores in general and at Eliot School in particular are 
below national averages. For example, the district mean on the word-meaning sec- 
tion of the Stanford Achievement Test at the end of the eighth month of instruction 
in the fourth grade (when students should have scored 4,8) was 3,6 in 1967, 4.0 in 
1968, and 4.4 in 1969 The scores for Eliot School for those three years were, respec- 
tively, 3.7, 3.9, 3.9. For the Spanish-speaking population of Eliot they were 3.2, 3.7, 
and 3.7. In 1969-70 about 70 percent of the Eliot students did not make a year’s gain 
for a year’s instruction in reading or mathematics. 

There has been a district-wide upgrading of skills. In reading, for all Title I 
schools in Gilroy in 1970, the second grade gained 6 months for 7 months instruction, 
the thii'd grade 7 months for 7 months instruction, and the fourth grade 3 months. 
In 1971 in all Title I schools, the gain in the second grade was 7 months in 7, in the 
third grade 4 months in 7, and in the fourth grade 3 months in 7. Thus, while the 
upward trend in achievement gain was sustained in the second grade this year, the 
third grade made less gain and the fourth grade showed the same gains as for 1970. 

In short, despite an upward trend in educational effectiveness as measured by 
standardized achievement tests, Gilroy students, and particularly the Spanish-sur- 
name students, typically test below national norms. GUSD officials have aggres- 
sively sought new programs to deal with this problem. In 1970 they experimented 
with performance contracting to see if it would solve Gilroy’s compensatory educa- 
tion problem. 
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m. THE GILROY WESTINGHOUSE LEARNING 
CORPORATION PROGRAM 



DEVELOPMENT OP THE PROGRAM 

Many performance contracting programB have involved need-assessment stud- 
ies, development of Requests for Proposals (RFPs), and other extensive planning 
activities. The Gilroy program stands in sharp contrast. The development of the 
program was simple, in part reflecting the style of the GUSD Superintendent, Dr. 
Robert Infelise. Infelise is a young activist and relatively new to the district. As he 
explained to the Hand researchers, he is inclined to get things goiiig and worry later 
about details, 

Gilroy’s entrance into performance contracting reflects this approach. Infelise 
was on a personnel recruiting trip to Albuquerque, New Mexico, when he first heard 
of performance contracting. Albuquerque was the headquarters of the Westing- 
house Liearning Corporation (WLC), and someone suggested that Infelise observe its 
activities. WLC was at that time considering a number of possible performance 
contracting programs^ as it turned out, it had five programs in operation during 
1970-71. Infelise visited WLC and discussed performance contracting with WLC 
Director Kenneth M. Kamerman. He shortly broke off* the discussion with the 
statement, ”Let’s not talk generalities, come to Gilroy and bid on a program.” 
Westinghouse and GUSD jointly worked out a program that was approved by the 
Gilroy School Board and embodied in a contract signed July 9, 1970 (see Appendix). 



fl 




Two points about the profram'S conception are worthy of emphasis. First, as 
Infelise made clear to Rand, he initially viewed the program as simply a remedial 
education effort. He is strongly committed to increasing the level of achievement 
scores in Gilroy; WLC asserted that it could provide a "catch-up” program and 
Infelise was willing to let it try. The use of performance contracting as an education 
change mechanism, or as a way of introducing accountability, or to achieve other 
goals that have played a part in other performance contracting programs, did not 
operate in Gilroy at the start. In Infelise’s eyes the only goal v^as to produce a big 
jump in Title I achievement scores. This point is important because given this 
conception, GUSD was less concerned than many other cities were with evaluations, 
teacher training, and other such ancillary activities. The point is also important 
because, as will be explained later, the program may turn out to have had some 
unintended but salutary curriculum benefits apart from compensatory education. 

The second point about the conception of the contract is that for both parties 
it was a launch into the unknown. WLC had not dealt with performance contracts 
before 1970-71; little of its past work had dealt with Spanish-speaking children; the 
program was a very small one — 103 students — and the economics of small perform- 
ance contracting programs were not well understood. For GUSD performance con- 
tracting was new. Of course, few school districts in the summer of 1970 knew much 
about the concept besides what they had heard about the 1969-70 program in Texar- 
kana. However, Gilroy had hit upon the idea of the performance contracting pro- 
gram late in the 1969-70 school year, wanted to implement it for the fall of 1970, 
and so had moved fast. The result was that the planning and structuring of the 
program became a joint and iterative process of exchanging proposals and counter- 
proposals between the two sides until an agreement could be reached. More impor- 
tant, many aspects of the program were left undefined. Throughout the year, many 
basic issues arose and had to be resolved in midstream. 



THE PROGRAM 

GUSD budgeted the program for $60,000, of which $48,000 came from Title I 
funds and the rest from the regular school district accounts. As the program was 
developed the contract itself became the major program document. Table 1 summa- 
rizes some of the major features of the program. 



PROGRAM AND RESOURCE INFORMATION 




Chapaatsx^iatias of students Grades 2-4; Title I; low socioeconomic 

status 



Progpcan aoope 

Class time 

Class size,., 

Number of sections 

Utilization 

Number of students 



1,25 reading , 1.25 math 
50 students per class 
2 

5 hours a day 
103 



FaoiUties 

Space 

Furnishings 

Staffing 

Special teachers 

Paraprofesslonals 

Equipment ■; 

Mate^tais 

Pp 0 - 80 m)ioe -Raining 

Inserviae training 

Other support 

Xno&nt%ue8 



2000 sq ft; 1 classroom j 1 activity area 
6 carrels, carpeting, tables 

1 full-tlffle-equivalent reading specialist 

2 per center, 1 per activity area 

Telex, tape recorders, cassette players, 
headset 

Books , games , toys 
4 days 

1 week, total 

Remote diagnostic and prescriptive 
25 per student—candy, scrip 



The contract as written called for instruction in two subjects, reading and math- 
ematics. The objective was simple: 400 achievement-years of gain— 200 in reading 
and 200 in math, or any combination thereof. 

The contract covered 103 students in grades 1 through 4. Since the target group 
consisted of all Title I students in Eliot School, elaborate selection criteria were 
unnecessary. 

The contract called for the students to be tested at the beginning of the program 
and assigned an "objective gain” (a concept that will be explained later). When TOC 
believed the student had achieved the specified gain, the student was to be retired 
with another form of the test, and if he had reached his objectives he was to be 
returned to his regular class. If he had not made the required score on the post-test, 
he was to continue in the program. 

If a student made less than one yearns gain between pre- and post-tests in a 
subject, no payment was to be made for that student. The upper limit of payment 
liability was set by the program objective gain determined by WLC individually for 
each student after the pre-testing and diagnostic testing had taken place at the 
beginning of the school year. * That is, it was not to cost the school still more if a 
student surpassed his assigned objective gain. Although the goal of the program was 
400 achievement-years, the agreement stipulated that Gilroy would enroll enough 
students to provide WLC an opportunity to obtain total objective gains of at least 
355 achievement-years. A clause in the contract also specified that the achievement- 
year would be gained in 90 hours of instruction or less. If the average time were 
greater than 90 hours, the price would be reduced proportionately. The contract 
states, "the objectives of the program are that all students enrolled in it will (a) 
advance at least one grade level in reading and math at the end of fiscal year 1971, 
and (b) will further progress to performance level at or near the grade level at which 
they are enrolled in school.” This implies average objective gains greater than one 
year in each subject. 

The significance of the requirement that enough students be enrolled to permit 
WLC to achieve a potential outcome of at least 366 achievement-years can be seen 
by recalling that the price per achievement-year was $168,76. Multiplying 168.76 by 
366 yields $69,906.25, compared to a budget for the project of $60,000.00. The figure 
of $168.76 was based on analysis by WLC and GUSD of the expected costs of the 
project. 



‘ For more discussion of the WLC gain cfbjective system, see Vol. 6 of this series, dealing with Grand 
Rapids, Michigan. 
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When the program was implemented two major changes were made. First, the 
program was limited to grades 2 through 4. Enrollment was sufficient in these grades 
to provide 103 students. Also, though nobody ever said so explicitly, elimination of 
the first grade simplified the problem of obtaining usable pre-test scores. 

The second major change was elimination of the variable exit-time feature. The 
students all remained in the program throughout the year. This change avoided the 
scheduling difficulties of returning students to regular courses at varying times, and 
avoided the question of how to find replacements for those who left. It also avoided 
the problems connected with the almost continuous testing program that would 
have been required if the contract had been carried out to the letter, 

WLC agreed to provide the school with a detailed description of the space and 
furnishings required and to assign a manager from its stafT. WLC also agreed to 
employ one or more aides in the center and to provide all training required for 
teachers and aides working in the program. WLC agreed to furnish all educational 
equipment and all educational and motivational materials required for use in the 
program, these to remain the property of WLC. 

G. USD agreed to make suitable space available for a Learning Center to accom- 
modate up to 62 students at a time. It additionally agreed to made adequate office 
space available for the use of the WLC stafiT manager and his secretary, and to 
provide all furniture for the Center and for the manager’s office. GUSD further 
agreed to select two teachers from its staff to wor k in the Learning Center, and gave 
WLC the opportunity to participate in and approve of their selection. The selected 
teachers were to be available for training at least two weeks before the start of the 
school year. 

WLC agreed that tho program would be ready to enroll students by September 
28, 1970, and the Center was to be opened and the program available to students 
at least 6 hours a day, 6 days a week during the school year. WLC agreed to accept 
for enrollment all students assigned to it by the school. WLC was to establish a 
learning objective and a program of study for each student, based on test informa- 
tion provided by the school. It reserved the right to notify the school within the first 
20 hours of any student’s attendance at the Center if it did not feel that the student 
could benefit from the program, and if, after review, GUSD concurred, the student 
was to be withdrawn from the program. WLC anticipated that not more than 3 
percent of the students would fall into this category. WLC further agreed to arrange 
in cooperation with the school for visitors, observers, orientation sessions, and 
teacher workshops. It further agreed to provide the school with appropriate informa- 
tion on the progress of each student enrolled in the program. 



The school agreed that in order to support the operation of the program it wou 
select Title I participants according to their need for remedial _ 

ematics and reading. GUSD agreed to pre-test each student “ J. J 

level using a nationally standardized test that reports in 
It agreed to enroll enough students to supply a potential total ®®® 

achfovement-years in the Learning Center, and to arrange for 103 ™® ■ ®" 
to attend the Learning Center for 2-1/2 hours every ^ay- 

assured a "standard minimum attendance” in the ^arnmg Center of at 
student-hours on at least 170 school days during this school year. 

The GUSD Director of Elementary Instruction, Mr. Rodney . y* _ 

mary cogni zance of the program as part of his 

ise took a deep personal interest in the program. No GUSD administrator was 
assigned full time to the project, however, nor was any managemen j. 

tractor or outside consultant involved. The basic philosophy, j 

and Kelley was that a very general agreement had been made between GUSD ®ml 
wil ProLms were expected to arise, but GUSD believed that good will on both 

sides would permit resolution on an ad hoc basis. 



EVALUATION 

Evaluation plans for the project developed gradually and informally. In the 

A nf 1970 when Rand asked Gilroj^ officials about their evaluation plans, they 

rk"rpoLtl;Th“^ "Trs 

A norm-referenced achievement test to be used for 

tobeselectedandadministeredby^^^ 

a goal of 400 achievement-years, and he telt that tne oniy however 

compare the actual test results against the 400 fi^re. »® 

that a nsvchologist intern who was working in the district planne ^ „inT,. 

tion of the program as his master’s thesis, and another district emp o^jee w 

ntog tu^re of the program data in a doctoral th^is. '®® ® 

Rand's studies in the district would contribute to the data base that would he 

available to him for an overall assessment of the program. hutconsider- 

Thus the evaluation was a relatively informal, mtradistrictaffai , ^ „ 

able^lSata weto generated. In onier to ‘^"h 

program, and to compare achievement gains made in the learning Center wit 



those made in other Title I programs^ the district administered both the Stanford 
Achievement Test, which was to be the basis for payment, and the Metropolitan 
Achievement Test, which Gilroy has used for some years in evaluating Title I 
programs. For some of the children in the program, then, two sets of pre- and 
post-test data were available. The MAT data provided a check on the program 
results as measured by the SAT data. The consistency of the two measurement 
instruments is an interesting matter, since ei recurring question about performance 
contracting is whether choice of a test makes a difference in apparent achievement 
gains. The Gilroy experience provides some evidence on this point and will be 
discussed later. The MAT data also permitted Rand to compare the WLC students 
with other Gilroy students. 

Various Gilroy school officials took an interest in assessing the results and 
impacts of the program. Several surveys were administered and, at the suggestion 
of Rand, the school psychologist administered the Gray Oral Reading Test (GORT) 
to some of the students in the program. Attendance data were collected, and Rand 
conducted a number of interviews v/ith project personnel. Thus, owing to various 
people's interest in the project, a considerable amount of data on program results 
was collected. In future programs, however, it seems unlikely that so much data will 
be produced adventitiously. If a school district wants to examine the broad effects 
of a perfc)rmance contracting program, it will be prudent to plan from the start for 
an evaluation. 

It should be emphasized that Gilroy differed from most programs in having two 
sets of standardized achievement test data as well as the GORT data. Administering 
two norm-referenced tests has the drawback that it takes time from the operation 
of a program and it increases the possibility of overtesting. Nonetheless, it has great 
merit. The pre- and post-test MAT data forearmed Gilroy with a way to confirm or 
deny a charge of teaching-to-the-test had such a charge arisen (which it did not). 
Also, administration of the MAT test even though the SAT was used for payment 
purposes means that the district’s longitudinal achievement-data series was not 
broken. Finally, and most important, administration of the MAT permits compari- 
son of the achievements of students in the WLC and in other Gilroy programs. 



IV. THE PROGRAM IN OPERATION 



Two GUSD teachers worked in the Center, each on a half-time basis. The teach- 
ers were what in California are called "Miller-Unruh” teachers, trained reading 
specialists who qualify to conduct special State-financed compensatory education 
program classes. GUSD also supplied support services from both building and dis- 
trict administration. WLC furnished all instructional materials and equipment, and 
conducted 30 hours of in-service training in September 1970 for the teachers and 
aides who were to participate in the program. 

The WLC Center began operating in the fall of 1970 with approximately 50 
students in the morning for 2-1/2 hours, and 60 in the afternoon for 2-1/2 hours. 
Each session had a teacher and two instructional aides. In previous WLC experi- 
ences— notably at the Center they ran in Albuquerque, which students paid to 
attend and normally came to after school — WLC had found that a higher staff-to- 
student ratio was necessary during start-up of the system than later on in the 
program. This was largely because students needed to learn both how the system 
operated and how to manage their own learning experiences. Despite this past 
experience, the program at Gilroy did not provide for additional personnel during 
the first weeks of operation, and as will be discussed later, this was unfortunate 
because some problems might have been avoided. The basic problem was that stu- 
dents needed to learn how the system operated. They were in an unfamiliar environ- 
ment where they were expected to work independently. Because of their need for 
advice and guidance, there were often long queues of children waiting for instruc- 
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tion. Teachers felt that students became frustrated by having to wait for a long time 
to have a lesson corrected or to receive a new assignment. The problem was exacer- 
bated for second-graders, who needed constant direction of their iGarning activities. 
The addition of one or two aides for a few weeks would have facilitated a smoothe*’ 
transition from a traditional setting to the modus operandi of the Learning Center. 
After about a month of operation, a third aide was assigned to the program so that 
two could help the teacher in the classroom and one could run the reinforcement 
center. 



FACILITIES 



GUSD made two rooms at Eliot School available to WLC: a large classroom 
facility and a room called the reinforcement center. Both were furnished according 
to specifications laid out by WLC. 

Eliot School is a Spanish-style building, well maintained and attractive, but 
several decades old. It has a central chamber onto which the school office and several 
classrooms open. The reinforcement center was constructed by partitioning the 
central chamber. The WLC classroom opened off the chamber and was next to the 
school office. The WLC manager and his secretary shared the office with the princi- 
pal, school secretaries, and a Telex machine that linked Gilroy with WLC’s Al- 
buquerque computer. The diagnostic work was done remotely by means of the Telex 
link. 

This was the best physical arrangement possible at Eliot without extensive 
structure modifications, but the facilities were not well suited to the WLC program. 
The basic problem was that they were too small. Center personnel had to deal with 
many problems due to overcrow^ding in the reinforcement center. For example, in 
November 1970 the children were complaining about losing time in the reinforce- 
ment center that they had earned. The original plan was for all children to start off* 
in the Learning Center and remain there until they began to earn time in the 
reinforcement center. This meant that the reinforcement center became very 
crowded toward the end of the instructional period. Children who earned their 'Tun 
time” too late might not be able to get into the center. 

One response to this situation would have been to let students carry over their 
reinforcement center "credits” until the next morning. WLC resisted this on the 
grounds that it would create too large a gap between the response and reinforcer. 
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Another obvious response would have been to reschedule students. Center per- 
sonnel suggested that instead of 50 students involved for 2-1/2 hours in the morning 
and 50 for 2-1/2 hours in the afternoon, the group be split into four equal sessions. 

The Eliot teachers not involved in the program resisted this suggestion, how- 
ever. They argued that the program had already caused substantial changes in their 
schedules and had disrupted their programs. Further disruption would be unfair to 
^hem and to the rest of the Eliot program. As a result, the original schedule was 
maintained until Easter, and the high noise level and other undesirable effects of 
the lack of space were accepted. At Easter, for reasons to be discussed later, there 
was a change in thinking on the part of the Eliot faculty and the reorganization into 
four groups took place. Facilities problems were greatly lessened. Nonetheless, the 
conventional California school building built in the 1930’s and 1940's is not well 
adapted to the type of program WLC conducted, and this was a handicap during most 
of the year. 
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CURRICULUM AND EDUCATIONAL PROCESSES 

WLC built the curriculum for its Centers by discussing with several school 
systems the textbook series generally adopted for classroom use in reading and 
mathematics. These series were then analyzed for identification of academic skills, 
the grade level at which these skills were introduced, and the development of the 
skill from its initial teaching to its more complex form. 

The instructional materials were selected by three criteria: (1) they should be 
self-instructional; (2) thfly should be usable on an individual learner-paced basis; (3) 
they should teach the skills identified in WLC's curriculum analysis. The materials 
selected included programmed instruction workbooks, audio tapes, film strips, and 
like materials. 

A student enrolled in a WLC Center took a standardized achievement test in 
reading and mathematics, end at the end of the program took an alternate form of 
the test. The gain between the two administrations of the test was ased as the basis 
for payment of that student’s achievement gain. Another purpose of the pre-test, 
however, was to give WLC an insight into where to start diagnostic testing for an 
individual student in order to determine his skill strengths and deficiencies. The 
results of the diagnostic tests were put oh the Telex and sent to Albuquerque, where 
they were used to select instructional sequences that WLC felt would best meet the 
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student's needs. An instructional sequence designed to teach a specific skill was 
often a combination of portions of available materials, and might employ several 
media. 

An instructional sequence was agreed upon by each student in consultation 
with the teacher. This took the form of a ^contract" for the student’s next unit of 
work. The Center sought to identify points of difficulty before a student lost much 
instructional time. The staff suggested more appropriate ways of reaching any 
particular student having difficulty. Each student experienced early and repeated 
success— an experience many of them had rarely enjoyed in a school setting. 

The WLC educational approach made considerable use of extrinsic incentives. 
In the Gilroy program two basic uses were made of this kind of motivational manage- 
ment. First, when a student successfully completed his contract, he was allowed a 
specific amount of time in the reinforcement room or, as it was also called, the 
activities area. In this part of the Center, students were allowed to engage in activi- 
ties different from the instructional activities that were taking place in the class- 
room. There were books and toys, including a small pool table. 

The second incentive war scrip money the students eai’ned by completing a 
series of instructional sequences. The WLC on-site manager arranged with several 
of th e merchants in town to accept scrip for hamburgers, milkshakes, movies, and 
some more material items from thedocal variety store and other shops. 

The rationale behind this kind of motivational management, as expressed, inter 
alia, in WLC publications, is that it will help a student to assume control over his 
own learning. In theory, this system generates a gradual shift from the usual class- 
room situation with maximum control of a student's instructional activity by a 
teacher, to minimum control by the teacher and maximum control by the student. 
The student is supposed to learn to verify his own progress by the test he takes after 
completing an instructional sequence, and learn the procedure to follow once a 
sequence has been completed so that he may start on the next lesson. Through this 
self-evaluati<jn procedure the student is supposed to learn to know his learning 
objective, how well he is succeeding in attaining it, and perhaps more important, 
perceive that the entire learning process consists of mastering a succession of rela- 
tively small steps. As noted above, however, many of the younger students were not 
able to direct their own learning effectively, in spite of the incentives. 

WLC's program reflected extensive study and'.design by educational researchers 
and curriculum authorities. It had been applied in various situations— for example, 
in WLC's Albuquerque Center, which offered remedial instruction on an after-school 



basis. WLC therefore anticipated no serious difficulties in implementing the pro- 
gram in Gilroy, but in fact, two major problems arose. 

The first was logistical. As previously discussed, the physical arrangements 
were not ideal. Also, implementing this type of program in the Gilroy context of 60 
students, a teacher, and an aide, turned out to have special characteristicis. In the 
early fall, students needed help with their lessons, or they needed to have assign- 
ments checked, progress tested, permission granted to go to the reinforcement cen- 
ter, and so forth. Sometimes they waited in queues for as long as half an hour, with 
resultant frustration for teachers and children alike. In Albuquerc|ue, WLC dealt 
with such problems by using extra personnel at the start of a project. In Gilroy the 
teachers had to "go it alone” and the logistics difficulties ofFset many of the potential 
advantages of the individualized nature of the program. 

The second problem was the appropriateness of curT*iculum and materials for 
the Gilroy situation. One aspect of this problem was expressed to us by a WLC 
official. He stated that WLC had not fully appreciated the special requirements 
imposed by the students’ being bilingual. He stated that if he had the program to 
do over he would have sought more specialized bilingual materials. 

A second aspect of this problem appears to have stemmed from the students’ 
being quite young. According to our observations, the materials seemed to assume 
that all the children were "ready” for subjects such as modern math. In fact, many 
of the students in the program had absorbed few of the concepts needed for math- 
ematics and reading skill instruction. Some of the children appeared to need a 
readiness program rather than a reading or math program. This view was supported 
by some teachers’ comments to us. 

A third aspect of the problem was that some of the Gilroy Center personnel 
questioned the WLC approach to minimizing the teacher’s involvement in the stu- 
dent-teacher-materials loop. They supported the concept of student control over 
their own learning, but felt that some of the children needed a more peraonalized 
approach. They believed that some students learned faster if the teacher worked 
with them. Toward the end of the year, we observed teachers and aides taking 
individual students into the on-site manager’s office for tutoring sessions. 

There are many pros and cons about whether or how the basic WLC program 
should have been modified. These are not important here; what is important is that 
both WLC and GUSD personnel believed that changes were required. Changes were 
accordingly made, but not early enough. In discussions with people involved in the 
program, the point that came across most forcefully was that the teachers should 
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have been involved earlier and more extensively in the design and implementation 
of the program. 



ESTIMATED PROGRAM COST 

It is hard to determine what this program cost or might cost in future years or 
in some other schoo. listrict. WLC costs are proprietary. Undoubtedly there were 
many start-up costs and learning costs that might be avoided in other years or 
programs. Consequently, the actual cost of the Gilroy program, even if it could be 
obtained, would not be very instructive for other school districts. 

A more helpful approach is to consider the resources involved in the Gilroy 
WLC Center and what these might reasonably be expected to cost. We have deve- 
loped such an estimate based on three assumptions: 

• that school space is available for the program at no charge; 

• that the diagnostic-prescriptive functions will be managed by the teacher 
in charge of the program; and 

• that management will be performed by the regular district staff at no 
increase in cost. 

Table 2 presents the estimated acquisition and operational cost of the program. 
The cost data are, for the most part, estimates of the program as conducted in jilroy 
in the 1970-71 school year, but the division of expense between the district and the 
contractor is not shown. The Telex remote diagnostic and prescriptive services that 
WLC used are not included. It is also assumed for this exercise that the on-site 
function of contractor management will be undert a ken by the regular district staff; 
therefore, no cost for general management is included. 

We estimate that a program like that in Gilroy in 1970-71 might be conducted 
for around $40,000. Dividing this by 100 students and two subjects each, this would 
be a cost per student per subject of about $200. Such a figure would be higher than 
is usual for conventional programs, but it is not out of line for remedial programs. 
Of course, cost per student is an input measure and is not very instructive. The more 
meaningful question is v/hether such a program produced sufficiently higher 
achievement gains to justify spending more than Gilroy would normally spend in 
conventional instruction. We will consider achievement in the next section. The 
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Table 2 

ESTBIATED PROGR^I COST 



Aoquieition aoet 
Program activities 

Implementation? 1 month planning j 

organization, scheduling 

Pre-service training: 2 teachers, 4 days 

Installation of equipment 

Equipment: tape recorders, cassettes, headphones 

Facilities: furniture (tables, chairs), carpeting, 

study carrels, storage space 

Materials 

Educational: books, tapes, filmstrips, etc 

Activities: ganes, toys, etc 

Total estimated acquisition cost 



$ 1,500 
600 
200 

3.000 

2.000 

3,000 

2,500 



$12,800 



Opevationat aosi 

Program activities: in-service training 

Salaries 



Specialists (1 fulltime equivalent) 

Paraprofesslonals (3) 

^taterlals 

Consumables, $10/student 

Incentives, $25/student 

Equipment 

ReplacMient (10% of equipment cost) 
ntenance 

Total estimated operational cost 
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Total eettmated ooate * . * 
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0 



14,000 

10,500 

1,000 

2,500 



200 

200 

$28,400 

$41 , 200 
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important point here is that performance contracting in Gilroy involved spending 
more than GUSD spends or their regular instructional program but not more than 
they spend on other compensatory programs. From Title I reports, we estimate that 
Gilroy spends in the neighborhood of $185 per student on remedial reading pro- 
grams and about $200 per student for remedial mathematics. In part, the high cost 
in the WLC Center reflected the small number of students in the program. Other 
performance contracting programs studied by Rand have somewhat lower costs 
because of economies of scale. The significant point for Gilroy is that the WLC 
program was more expensive than ordinary classroom instruction but not out of line 
with the cost of other remedial programs. 



TESTING 

We made a special study of testing procedures because contractors are paid on 
the basis of achievement scores, and because in Gilroy two kinds of achievement 
tests were administered on a pre- and post-test basis. The SAT was administered by 
teachers on September 23, 1970, and we sat in the room while the second and fourth 
grades were tested. We also observed the B.IAT administration by a psychometrist 
on October 7 and 8. 

In general, what we observed raised serious questions about the reliability of 
achievement test data for young student populations such as those in the Gilroy 
project. We will not review here the philosophical and statistical questions that have 
been raised about the reliability or interpretation of gain scores for performance 
contracting purposes. (GUSD officials understood these questions and were con- 
cerned about them. One administrator discussed with us at length his reservations 
about administering achievement tests to selected populations of students who fall 
on a "tail” of the distribution of the population on which the test was normed.)® The 
mechanics of test-taking by populations such as those in the Gilroy program, how- 
ever, lead us to suspect that there must be a very large error component in any 
individual student’s score. 

The test administrators had a difficult time following the specified standard 

" For mors on testing issues, see VoL 1 of this aeries, M. B. Carpenter and G, R. Hall, Case Studies 
in Educational Per^rmance Contracting: L Conclusions and Implications^ The Rand Corporation, R* 
900/1-HEW, December 1971, and references cited therein. 



procedures for the instruments. They had to spend much of their time controlling 
the children and trying to hold their attention. In some cases the administrators felt 
it necessary to deviate from the prescribed timing or standard text. The mechanics 
of taking achievement tests seemed to confuse many students, who apparently 
resorted to filling in answers without pondering over the questions. 

To illustrate, we followed the eye movements of one youngster who never read 
the stem of the question, but simply marked the answer columns. During a break 
in one of the testing sessions we talked with several of Liie children about the test 
and what they were doing. Because they did not understand the instructions, or 
because they could not understand the stems of the items, they made up their own 
rules for the "game.” One child marked the one word he understood out of the four 
alternatives; another chose and followed a pattern whose rationale escaped us, if it 
had one at all. 

Such problems are to be expected, The Gilroy pupils were quite young, many 
of them had a limited command of English, and they were not able students. Con- 
fronted with instructions they found murky in a situation with no compelling inter- 
est for them, they devised impromptu solutions that worked for them, however 
frustrating they were for school authorities, contractors, and researchers. 

' Standardized tests were not designed to serve individual diagnosis or to evalu- 
ate the efficacy of instruction with respect to individual students. They were de- 
signed for analyses in which mean scores are adequate and the unreliability of 
individual scores presents no major difficulty. If performance contracting programs 
continue to involve students with learning and language problems, however, the 
problems of testing and test administration for such populations will have to be 
addressed. Perhaps the answer is to devote more painstaking care to the form and 
phraseolo^ of instructions or perhaps to devise entirely new types of instructions. 
Perhaps the answer is a new type of test altogether. We are not able to prescribe 
a remedy but the problem is serious, as the Gilroy experience shows. 



V. ACADEMIC ACHIEVEMENT 



READING ACHIEVEMENT 

Between the pre-test in October and the post-test in June, in the second grade 
there was three months growth, in the third grade five months, in the fourth grade 
seven. (We are using mean scores, which avoid some of the statistical error problems 
discussed in Sec. IV. Even so, the statistical meaningfulness of these test results 
remains in question considering the problems of obtaining gain scores in general and 
for populations such as the Gilroy students in particular.) Data on the tests are 
presented for both mean scores and median scores in Table 3. 

All gains must be considered in relation to a standard or some comparison 
group. Compared with WLC’s initial expectations, the gains were certainly disap- 
pointing. WLC expected at least one year’s gain per student. Some students made 
such gains, but most did not. In comparison with the rest of the district’s Title I 
program, however, the WLC students did better.® On third-grade students 

in the Center gained six months, while the rest of the Title I students gained four 
months; in the fourth grade, the WLC students gained seven months, the rest three. 
The WLC program accomplished more than other programs aimed at the same 
population in the district. 

We were interested in determining if the student’s entering achievement scores 
“ GUSD Annual Evaluation Report, ESEA Title 1, July 15, 1971. 




Table 3 

RE^^ING ACHIEVEMENT IN WLC AS IffiASURED BY SAT 





Mean Scores 


Median Scores 


Grade 


Pre-test Post-test 


Gain 


Pre-test Post-test 


Gain 


2 


1.4 1.7 


0.3 


1.4 1.6 


0.2 


3 


2.1 2.6 


0.5 


1.9 2.7 


0.8 


4 


2.6 3.3 


0.7 


2.6 3.3 


0.7 



affected the amount gained. Did students with low pre-test scores gain more than 
students with high test scores? Such a result might have been expected if the 
regression-to-the-mean effect was an important influence. To answer this question, 
we divided each class into thirds based on their pre-test and computed gains for each 
of the subgroups. There are differences, but they are neither consistent, as can be 
seen from Table 4, nor statistically significant as determined by a t-test. This result 
implies that regression to the mean does not explain the gain scores. 

Another question that deserved consideration was whether there were ^ rub-off’ 
effects from the WLC Center on the achievement of the students in Eliot School who 
did not participate in the WLC Center (nonprogram students). Nonprogram students 
in Eliot are really a different population from the program students, because nor- 
mally those students with the greatest educational deficits are selected to partici- 
pate in Title I programs. The means of the pre-test achievement scores for Title I 
and for other Eliot students are significantly different at the .01 level of significance 
as measured by the t-test. Table 6 lists the pre-test means for the two groups. As 
would be expected, the Title I students participating in the WLC Center started with 




Table 4 



READING ACHIEVEJffiNT BY raiRDS IN WLC 
AS MEASURED BY SAT 



Grade 


No. of 
Students 


Thlrda 


Pre-test 

Mean 


Post- test 
Mean 


Galnb 


2 


10 


1 


1.2 


1.5 


0,3 




14 


2 


1.4 


1.6 


0.2 




9 


3 


1.5 


1.9 


0.4 


3 


12 


: 1 


1.7 


2.3 


0.6 




12 


2 


2.0 


2.4 


0.4 




10 


3 


2.8 


3.3 


0.5 


4 


8 


1 


2.2 


2,9 


0.7 




7 


2 


2.6 


3.2 


0.6 




7 


3 


3.0 


3.8 


0.8 



g 

The figure 1 represents the lowest third of the 
grade in entering reading scores. 

As of June testing. 



lower mean scores than those of the nonprogram students at Eliot. Furthermore, 
because the Center absorbed so many students, the nonprogram students enjoyed 
the windfall advantage of very small classes. It therefore seemed possible that 
decreased class size as well as other possible influences might have affected nonpro- 
gram student achievement. 

The pattern of gains for the two groups, shown in Table 6, does not show any 
consistent difference, however. While the nonprogram students made greater gains 
in the second grade, the WLC students did better in the third grade. Small class-sizes 
yielded no notable achievement-score benefits to the nonprogram students. 

Table 6 has an implication for. interpreting the outcome of the program. Exami- 
nation of the pre-test scores indicates that the students placed in the WLC Center 




Table 5 



READING ACHIEVEtffiNT OF WLC AND NONPROGR^ STUDENTS 
AS tffiASURED BY SAT 





WLC 


Nonprogram 


Gradt, 


Pra- test 


Post-test 


Gain 


Pre-test 


Post-test 


Gain 


23 


1.4 


1.7 


0.3 


1.9 


2.6 


0.7 


3 b 


2.1 


2.6 


0.5 


3.1 


3.3 


0.2 


4 C 


2.6 


3.3 


0.7 


3.7 


4.5 


0.8 



^Slinlficant at the .01 level. 
^Significant at the .04 level. 
^Not significant. 



had been steadily falling behind their Elliot School cohorts as their schooling pro- 
gressed. The WLC program may not have enabled the Title I students to catch up, 
but there was no appreciable widening of the gap in grades 3 and 4, although the 
gap widened in grade 2. 




MATHEMATICS ACHIEVEMENT 

WLC had greater mathematics achievement gains than reading gains, but the 
differences between the gains in the two subjects were not staiiswicahy significant 
(by t-test, using the ,06 level), Only the fourth grade achieved the objective of one 
achievement-year per calendar year. Using mean scores, the second grade gained 
five months on the SAT, the third grade seven months, and the fourth grade 1.1 
years. Table 6 shows both mean and median data. 

A comparison of the gains made in mathematics by the students in the WLC 
Center and by other Title I students presented an anomaly: gain was one year for 




Table 6 



MATHEMATICS ACHIEVEMNT IN WLC AS MEASURED BY SAT 





Mean Scores 


Median Scoreii 


Grade 


Pre-test Po 


st-test 


Gain 


Pre-test Post"- test 


Gain 


2 


1.5 


2.0 


0.5 


1.5 2.0 


0.5 


3 


2,1 


2.8 


0.7 


2.1 2.6 


0.5 


4 


2.5 


3.6 


1.1 


2.4 3.2 


0.8 
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the district as a whole in the third grade; for the WLC Center students it was seven 
months. In the fourth grade, on the other hand, the gain for the district as a whole 
was six months; for the WLC Center students it was 1.1 years. The results suggest 
—perhaps speciously, perhaps not— that the district’s program is peculiarly effec- 
tive for the third grade and WLC’s for the fourth. 

We analyzed the differential effect depending upon a student’s entering achieve- 
ment score. The evidence is even stronger here that the gains achieved are not 
artifacts of the regression-to-the-mean effect. No significant differences in amount 
of gain can be attributed to entering scores (as determined by a t-test using .05 level 
of significance). In fact, in every grade students with the highest entering achieve- 
ment scores gained bp much as or more than did students with lower scores, and the 
gains in the fourth grade are the largest for all groups. The data are presented in 
Table 7. 

We also compared the gains in mathematics for students in Eliot School w ho did 
and did not participate in the WLC Center. They proved to be nearly identical (see 
Table 8). Again, the nonprogram students both started and ended the year with 
higher achievement levels. Thus, in mathematics as in reading, the extra resources 
made available by existence of the WLC Center at Eliot do not seem to have measur- 
ably benefited the nonprogram students. There is also a bright side for the WLC 
students: as in reading, the differentials between the WLC students and other stu- 
dents at least did not widen in 1070-71. 



K. } 





I 



Table 7 



MATHEMATICS ACHIEVEMENT BY THIKDS IN WLC 
AS MEASURED BY SAT 





No. of 




Pre-test 


Post-test 


7. 


Grade 


Students 


Thirda 


Mean 


Mean 


GalnO 


2 


9 


1 


1.2 


1.7 


0.5 




11 


2 


1.4 


1.9 


0.5 




13 


3 


1.7 


2.3 


0.6 


3 


10 


1 


1.5 


2.1 


0.6 




11 


2 


2.1 


2.6 


0.5 




13 


3 


2.7 


3.5 


0.8 


4 


7 


1 


1.9 


3.0 


1.1 




7 


2 


2.4 


3.5 


1.1 




8 


3 


3.0 


4.1 


1.1 



®The figure 1 represents the lowest third of the 
grade In entering mathematics scores. 

^As of June testing. 



IMPLICATIONS 

If program success is measured against the standard announced at the begin- 
ning of the program- -one achievement-year gain per student in reading and in 
mathematics— then the program was a failure. If the standard is an improvement 
in the scholastic achievement of Title I Gilroy students, then the program looks 
more favorable. The pupils in the WLC Center generally did better than similar 
Gilroy students not in the programs, and they recorded gains similar to those of 
non-Title I students at Eliot. 

Two considerations forestall any easy judgments, however. First, the WLC pro- 
gram cost more than the usual- Gilroy instructional system. Perhaps had more 
money been spent in conventional ways, as good or better results might have been 
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Table 8 

MATHEMATICS ACHIEVE^ffiNT OF WLC AND NONPROGRAM STUDENTS 

AS IffiASURED BY SAT 





m.c 


Nonprogram 


Grade 


Pre-test 


Post-test 


Gain 


Pre--teet 


Post-^test 


Gain 


2 


1.5 


2.0 


0.5 


i.9 


2.6 


0.7 


3 


2.1 


2.8 


0.7 


2.7 


3.4 


0.7 


4 


2.5 


3.6 


1.1 


3.2 


4.3 


1.1 



achieved; perhaps not. Since there is no equal-cost conventional program to compare 
with the WLC achievement, one cannot make cost-effectiveness comparisons. All 
one can say is that the WLC program both cost more and produced somewhat higher 
achievement scores. 

The other consideration is that 1970-71 was the development year for the WLC 
project in Gilroy;, if the Center had been continued, future years might have seen 
higher gains. As discussed previously, there v/ere significant changes in s(:heduies, 
curriculum, and procedures during the yeai% Perhaps they might have produced 
more substantial gains in future years. This possibility suggests the advantage of 
multiyear programs and the prematurity of evaluating a program on the basis of a 
siiigle year’s results. 



THE GENERALIZABILITY OF ACHIEVEMENT GAINS 

It is a perennial and still unsettled issue whether reading-achievement scores 
produced under any sort of program, including performance contracting, bear a 
direct relationship to the skills necessary to do ordinary schoolwork. When we raised 
this issue, GUSD expressed a willingness to cooperate in an effort to obtain some 
data on this point. The school-district psychometrist and the principal of Eliot School 
agreed to administer a small number of individual reading tests. The test selected 
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was the Gray Oral Reading Test (GORT), It is a relatively short test, takin^^ about 
IB or 20 minutes to administer. The objective was to see how students with various 
achievement gains as measured by the usual norm-referenced tests would score on 
a different type of reading test, one possibly more related to the type of exercises they 
might perform in a classroom. 

The GORT was to be administered to 4 students from each grade— 2 with high 
and 2 with low achievement scores on the SAI’. The scores on the two tests are shown 
in Table 9 (one high-scoring fourth-grader was omitted from the table because his 
SAT score was missing). The choice of subjects was left to the psychometrist.* In 
defining low and high, he used the results of the SAT administered at the end of the 
school year. Figure 3 is a graph of the SAT and GORT scores. While we did not 
compute the correlation because of the small number of observations, we have 
drawn the line on the figure representing a perfect correlation between the two 
scores. We interpret the relationship between the test scores as indicating that both 
tests do tap many of the same skills. A student who does well on the SAT, which is 
a group paper-and-pencil test, is also likely to do well on an individually adminis- 
tered test in which he demonstrates his oral reading proficiency for an examiner. 
Of the li scores, 10 shov/ a very good correlation between the two tests. One pair 
of scores (3H) has an unusually large spread. The relationship among the other 
scores is so consistent that we wonder about the discrepancy. One possible explana- 
tion is that the GORT is not scored for comprehension although the SAT is, and that 
the comprehension questions lowered the student's SAT score.® 

All in all, it would appear from this small sample that the SAT is a reasonable 
proxy for an elementary student’s ability to read as measured by individually ad- 
ministered oral tests, and that it is plausible that the performance contract in 



The sample he selected contains some ambiguities. For example, one student who was chosen as a 
low-scorer from grade 4 had a higher score on the SAT than did another student who was designate..! as 
a high-scoring fourth-grader. Going back to the original data in ftn attempt to reconstruct the way in 
which students were chosen did not yield a systematic explanation. It is probably of no great import 
because there was no systematic selection of students to take the OORT. On the other hand, it leaves 
us in the position of saying somewhat reluctantly that we cannot adequately describe these students 
either as a sample who scored high or low on the SAT or a" a sample who made good or poor gains on 
the SAT. Nevertheless, the data support the notion that for those students tested both of these tests are 
measuring reading skills. There is no justification, hoWever, for suggesting that any other two sets of 
scores would produce the same kinds of results or that another sample would. It should also be pointed 
out that the standard error of meafiuremant of the SAT reading test is larger than the difference between 
the SAT and the GORT scorep,. 

* O. K. Buros (ed.). The Sixth Mental Measurement Yearbook, The Gryphon Press, Highland Park, 
New Jersey, 1966, p. 842. 



Table 9 



GORT AND SAT SCORES FOR A S^LE OF STUDENTS 



Student 


SAT 


GORT 


2L 


1.4 


1.1 


2L 


1.4 


1.3 


2H 


2.2 


2.3 


2H 


3.4 


3.0 


3L 


1.8 


1.7 


3L 


2.0 


2.0 


3H 


2.7 


3.2 


3H 


3.9 


6.9 


4L 


2.5 


1.9 


4L 


3.7 


3.2 


4H 


2.9 


2.6 



reading improved the students^ reading ability and not some other set of related 
skills. Of course the small sample and the large standard error of measurement for 
the SAT data mean that these results must be regarded as no more than suggestive. 
They do indicate the feasibility, in performance contracting, of examining the de- 
gree to which achievement gains on standardized tests correlate with other meas- 
ures of ability. They also suggest that norm-referenced tests may be better proxies 
for achievement than many have believed. Unfortunately, any firm conclusions on 
these topics require more data. 



DOES THE CHOICE OF AN ACHIEVEMENT TEST 
MAKE A DIFFERENCE? 

If measurements of achievement-gain vary according to which te&t is adminis- 
tered, the phenomenon is obviously important for performance contractors, who are 
paid on the basis of these measurements — ^and for the school districts that pay them. 
This is another unsettled issue that has generated more argument than empirical 
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Fig. .3 — GORT and SAT scores for a sample of students 



data; it would be helpful for everyone concerned if more schools conducted parallel 
tests with the same students, as Gilroy did. The Gilroy data do not resolve this issue, 
but they provide a further bit of evidence. 



As discussed, the SAT was used as the basis for contractor payment, and partici- 
pants in the WLC Center also took the MAT, the district Title I test. Table 10 shows 
for the third and fourth grades, the pre-test means, the post-test means, and the 
gains on both tests for those students in the program for whom we have both sets 
of test scores. The gain in the third grade was five months on the MAT and four on 
the SAT; in the fourth grade, the gain was seven months on both tests. 






t-gains 



At least in this case there is no substantial aifference in the 
reflected by the two tests, although the pre- and post-test scores are not the same. 



Table 10 

COMPARISON OF WLC READING GAINS ON MAT AND SAT 





MAT 


SAT 


Grade 


Pre-test 


Post-test 


Gain 


Pre-test 


Post-test 


Gain 


3 


2.4 


2.9 


0.5 


2.3 


2.7 


0.4 


4 


3.0 


3.7 


0.7 


2.6 


3.3 


0.7 



NOTE! These scores differ from Chose reported elsewhere 
because this analysis Includes only students for whom we 
have both sets of test scores. 





VI. OTHER ASPECTS OF THE PROGRAM 



This section discusses the reactions of students, parents, and GUSD personnel 
to the program. 



STUDENT REACTION 

GUSD project personnel conducted 36 interviews with students in the program. 
A standard form was used but not all children responded to all questions. The first 
question was, "What did you like about Westinghouse?” In answer, 10 students said 
they liked the coupons or reward scrip the most, 8 liked the activities area, 6 liked 
reading books, 3 liked the math instruction, 2 the candy, 1 the tests, and another 
liked the teachers. 

The second question was, "What didn’t you like about Westinghouse?” The 
largest number of students, 6, responded "noise,” and 3 students said it, was too long. 
Concerning the "buz^ break,” the 6-minute period in the activities area, 33 students 
said they liked it, 3 that they did not. In response to the question, "What did you 
think about using tape recordings and tapes to learn lessons?,” 22 students were 
favorable and 10 unfavorable. 

Another question was, "How did you like going on shorter sessions?” This ques- 
tion referred to the change in the program instituted after the tests at the 120-hour 



point, when instead of having 60 students in 2-1/2-hour sessions, each session was 
divided into two and 26 students at a time went to the Center for 1-1/4 hours. The 
response was largely in favor of the shorter sossions; 24 students said they preferred 
them. When asked why , their responses were that they did not get as tired, it was 
quieter, and they got more help. But 8 studcni-i said they preferred the longer 
sessions because they had more worh: time and got more done. 

Since only about 30 percent of the students were interviewed, and we are not 
sure they were selected randomly, we would hesitate to extrapolate these answers 
to the student population as a whole. Nonetheless, this sample of students seems to 
have liked the program and the reward system. 



STUDENT ArrENDANCE 

It is extremely difficult to measure a program’s emotional and nonacademic 
effects. Performance contracts have so far been applied largely to reading and 
mathematics, partly because there are standardized and accepted measuring instru- 
ments in those subjects. On the other hand, all teachers, administrators, and firms 
that have entered into performance contracting have expressed concern about the 
long-term effects on students. Something is lost if a child is jockeyed into a dramatic 
increase in readir g ability if in so doing he learns to dislike reading. Consequently, 
everyone concerne ^ with innovative programs is concerned about measuring their 
noncognitive effects. 

Such measurement is beset with pitfalls. Questionnaires are often used, but one 
needs to be wary of student responses to questionnaires, particularly in a school 
setting. Students learn conformity in most school situations, and tend to believe 
tenaciously that there is a "right answer” for every question. Even on a question- 
naire, they tend to cast about for the elusive right answers or to answer demurely 
what they think the teache r wants to hear. Because of this, administering question- 
naires about attitudes to an eiementary population, and especially to students in the 
early grades, leads to questionable estimates of effects on children’s attitudes. 

For this reason, there has been a search for unobtrusive quantifiable data that 
could be used as a surrogate for attitude change. Unfortunately', there are no unob- 
trusive measures that are obvious and good proxies for noncognitive effects. Attend- 
ance is often used as a measure, on the basis that unhappy students are more likely 
to play truant than are happy students. Yet, attendance may be controlled more by 



family necessity than pupil choice, and factors other than the student’s views about 
a particular course or program may motivate his absences. Nonetheless, for want 
of a better alternative, attendance continues to be used as an index of affective 
change. 

We were able to get attendance figures for project participants in Gilroy during 
1970-71, when they were in the second, third, and fourth grades, and also for the 
previous year when they had been in the first, second, and third grades. Table 11 
presents the data. Absenteeism decreased in the third grade; it increased slightly in 
both the second and fourth grades. (None of these differences are statistically signifi- 
cant.) Without explanatory historical data for each student absence, there is no basis 
for imputing changes in attendance patterns to the effects oi the program. The 
appropriate conclusion is that the Gilroy attendance figures do not indicate much 
of an affective impact— either positive or negative^ — on students. 



Table 11 

ATTENDMCE IIGURIS FOR WLC PARTICIPANTS 







1970-71 




1969^70 






Average 




Average 




No. of 


Absence 




Abffsnce 


Grade 


Students 


per Student 


Grade 


per Student 


2 


32 


12.0 


1 


11.5 


3 


33 


10.0 


2 


11.8 


4 


24 


8.0 


3 


7.8 



PARENT REACTION 

At the end of the program a short questionnaire in both English and Spanish 
was sent home to the parents of children who had participated in the program. 
About half responded. 

The ovei*whelming majority, 48 out of 68, were pleased with their children’s 
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experience in the Center, liked the reward system, and indicated that their children 
liked the reward system. Other questions elicited more variation in response. Not 
all parents felt they could judge their children’s progress as compared with the 
previous year. Of those who did answer the question, the large majority, 34 out of 
44, believed their children progressed more than they had last year. In response to 
the question, "Do you think your child learned more because of the reward system?” 
two-thirds of the parents answered "yes.” Most said their children were happier in 
school in 1970-71; only 3 of 49 parents answering the question believed their children 
were less happy. 

All in all, the responding parents seemed quite pleased with the program. There 
was a space on the questionnaire for parents’ comments. Not many parents used it, 
but of those who did three commented that their children had a new interest in 
learning. Two others were more specific; one said, "My child wants to spell every 
word he sees,” and another said, "When we are travelling my child wants to read 
every sign on the road.” One parent said her child expressed more interest in 
learning arithmetic, her weakest subject. One comment was that the teachers also 
seemed more interested in teaching, and that this of course was good for the chil- 
dren. A few parents commented on the reward system, observing that rewards 
stimulated their children’s interest in learning, or made them want to do better 
work, or spurred them on to complete their work. Of the two negative comments on 
the reward system, one parent simply said, "My son was unhappy. He did not care 
about the rewards, ’ and the other said, "My children liked the rev;ards but not 
enough to work hard all the time.” Two comments were interesting because contra- 
dictory; "The program was good because no pressure was put on the children,” and, 
"The program was not demanding enough.” 

It is anyone’s guess whether these responses also reflect the opinions of parents 
who did not answer the questionnaire. One suspects that the respondents may have 
been motivated by a healthy bias: since most of them were in favor of the program, 
they may have filled out the questionnaire in hopes that their support would ensure 
the continuance of such programs in the future and that their children would again 
have an opportunity to participate. On the other hand, it would have been reasona- 
ble to expect numerous negative responses if dislike for the program had been very 
common among parents. Apathy and reticence were probably more prevalent than 
active opposition. 
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DISTRICT PERSONNEL REACTIONS 



To examine teachers’ and administrators’ reactions ro the WLC Centerj we 
conducted a series of interviews during the school ye^ r and at its close. We talked 
with GUSD officials, including the superintendeut and the administrator responsi- 
ble for overseeing the program, the administrator in charge of all special programs, 
the principal of Eliot School, the psychometr ist and psychologist attached to the 
district office, the teachers and instructional aides in the program, and teachers in 
the school who did not participate in the program. Without attributing statements 
to particular people, we present a summary of opinion. 

E"^eryone was asked at the end of the year, ''What would you do differently if 
you had it to do over again?” The first response of all the people we talked with was 
always the same: that they would get the teaching stafiT involved earlier. As dis- 
cussed in Sections III and IV, the program underwent a number of significant 
changes in its organization and conduct. Teacher inputs to the change-decisions 
were important. Looking back, however, the consensus was that more and earlier 
teacher involvement would have been very advantageous. 

There were some tensions at Eliot School between the program staff and the 
nonprogram staff*. Nonprogram teachers found it disruptive to have students going 
in and out of the classrooms to attend the Center. It made scheduling lessons and 
carrying out many classroom activities difflcult. The tension among the Eliot staff 
lessened as the year wore on, however, and disappeared by the time the results of 
the 120-hour testing had been released. The testing results made it apparent that 
the students in the program had not made unusiial gains. Apparently, the nonpro- 
gram teachers had felt threatened by the WLC Center because, whether it was 
stated or not, they knew they were going to be compared with another— and outside 
— ^group, brought in to raise student achievement, and therefore bringing with it the 
implicit notion that the teachers had been tried and found somewhat wanting. True, 
the program teachers were not outsiders to the system; they had previously been 
GUSD Miller-Unruh reading specialists. The introduction of WLC into the district, 
however, evtm using local teachers, could not help but set up some degree of rivalry 
between "insiders” and "outsiders.” 

After the 120-hour results had been released, it was possible to institute a 
change in the program that the program teachers and the on-site director had 
wanted to institute much earlier in the year but that the nonprogram teachers had 



not accepted. This was the previously discussed shift from 2 to 4 sections, proposed 
in the fall by the WLC Center staff but opposed by the nonprogram staff on the 
grounds that there had been too many disruptions to their programs and they could 
not accommodate to more. After the test results appeared, the other teachers were 
more willing to inconvenience themselves for the benefit of the Center. 

As the year progressed, both program and nonprogram teachers began talking 
about operating their own learning center in the school. They were beginning to 
accept the kind of instruction taking place in the WLC Center and to think about 
how they would run such a program without an outside agency in the school. There 
was more and more talk among the teachers of adapting the WLC Center to district 
operation on a continuing basis. In the spring some actual proposals were made. The 
possibility of Center continuation under teacher sponsorship will be discussed fur- 
ther in the next section. 
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VII. CONCLUSIONS 



It would be simplistic to dub the program either a success or a failure. The 
answer to this question has several dimensions, some negative, some positive. From 
the viewpoint of WLC, the program must have been a financial disappointment. The 
details of the financial settlement are under negotAation between WLC and GUSD, 
and how much WLC will be paid is unknown.® The interpretation of the achievement 
results and the financial implications of tne program changes apparently are at 
issue. It is clear from the final test data that paymerit for achievement will be 
considerably less than the maximum payment possible, which required students to 
make either one year’s gain per subject per student or achieve what the contract 
calls the objective gain. This latter, the reader will recnll, is a goal specified by WLC 
for each student based on the rBsults of the diagnostic testing carried out at the start 
of the program. In general, the objective-gain computation represented achieve- 
ment-level goals for the end of the year closer to national norms than normally 
would be expected for these students or than had been made by Gilroy Title I 
students in the past. The minimum objective gain for any student was 1.3 year’s 
growth. 

* The sparsanesi of information on the outconia of Gilroy and other performance contracting pro- 
grams merits a comment. Educators are used to open dissemination of data ahout experimental programs. 
Because performance contracting programs entail financial settlements* however, it is likely that results 
will not be made public until all claims have been resolved. This may tal e years if the chdms are under 
litigation. 



How many of the students qualified for achievement payments as measured by 
the final test has not been announced. On the basis of thw; test results at the end of 
120 hours of instruction (around Easter), it would be surprising if many did. On the 
120-hour test only 6 percent of the students had gained a year in reading and only 
16 percent had gained a year in mathematics. 

In the spring of 1971, WLC discontinued its school operations division. Gilroy 
was only one of several WLC programs, but it seems reasonable to suppose that the 
financial outcome in Gilroy contributed to WLC’s decision. 

From the standpoint of the academic aspirations of both WLC and GLTSD, the 
results were also discouraging. Both had hoped that the program would solve Gil- 
roy’s compensatory education problems, and bring Title I students up to national 
achievement-test norms. It clearly did not do this. 

It world be wrong, however, to write off the Gilroy program as merely another 
failed experiment. First, the WLC students surpassed their other Gilroy cor ff.er- 
parts, and the gap between the Title I and other students at l^liot at least d ot 
widen in grades 3 and 4, Second, the Gilroy program evolved through the year; gains 
might be greater under some steady state in the future. 

Even apart from these considerations, the program seems to have served as an 
agent for curriculum ch,.4nge in Gilroy. The staff at Eliot School became very inter- 
ested in the WLC activities. As the year progressed there was a great deal of coopera- 
tion between the on-site director and the teachers in the WLC Center. The teachers 
made suggestions about program changes and additional material, most of which 
WLC accepted. 

Since WLC has discontinued its school operations division, it cannot maintain 
the Center in Gilroy. But the teachers at Eliot School, both those who had been in 
the program and those who had not, submitted a proposal to the Superintendent to 
run their own reading resource center in 1971-72, largely modeled after the WLC 
Center. The district has some discretionary funds available, but this proposal will 
have to compete with other proposals from other schools that also want to run 
innovative programs. As of this writing, the proposal has not yet received official 
approval but the teachers’ interest has not flagged; they hope to win flnal approval 
in 1972. Whether or no^^ the new center is funded, the performance contracting 
experience of 1970-71 has apparently inspired the Eliot teachers to develop an 
individualized program of instruction. 

In sum, the Gilroy program had three important outcomes. First, the students 
did not make the large achievement gains that n’culd have brought financial reward 



to WLC and permitted GUSD to lessen the gap between reading and mathematics 
scores at Eliot and its non-Title I schools. Second, the students at WLC center scored 
achievement gains somewhat higher than those of other Title I students. Third, the 
program created interest among Gilroy teachers in intra-GUSD applications of the 
Westinghouse technology for a more individualized instruction. 

The Gilroy program also points to two general conclusions that are significant 
for the planning, management, and evaluation of performance contracting pro- 
grams in general. First, it is unlikely that life will ever be so simple as to permit 
a school or contractor to set a fixed and complete system in motion and then measure 
the extent to which it meets clearly specified objectives. Instead, they are more likely 
to find th mselves managing a constantly changing program and trying to judge the 
outcomes of the evolution. This conclusion has several implications. 

First, it is important that the written contracts contain provisions that allow 
flexibility for needed changes. Programs like the one in Gilroy usually start off in 
a warm atmosphere of mutual good will, witli the good-natured expectation that the 
contractor and the school district will be able to iron out, as they come up, difficulties 
that will affect the contractor's payment. Very often, this expectation is overly 
sanguine. In Gilroy, as in other programs, the contract is yet to be settled and data 
are therefore kept confidential. The obvious but specious implication is that both 
parties should spell matters out in meticulous detail at the start of a program, but 
such advice is a cuunsel-of-perfection and woulf probably be counterproductive if it 
froze lut the flexibility needed for healthy changes along the way. The Gilroy 
program, and many others, demonstrate that there is no off-the-shelf program that 
can be implemented without substantial change. Clearly, provisions for change must 
be made, and change is sure to have financial implications. I'he best strategy may 
be to expect problems in resolving the financial outcomes of programs, and establish 
in advance at least the procedures to follow for instituting changes and dealing with 
the altered financial liabilities they entail. 

The likelihood of program change has two other implications, both discussed 
previously. One is the advantage of early involvement of teachers in program devel- 
opment and implementation. The other is the advantage of multiyear programs over 
single-year programs. The Gilroy experience indicates that it is perhaps too much 
to ask either a contractor or a school to devise, within a single year, a well- 
articulated program nicely calibrated to the district’s needs and resources. 

The second general conclusion emerging from the Gilroy program is that per- 
formance contracting is likely to have multiple outcomes, and the value of the 



program is likely to depend on the relative weights assigned to these nutcoi'^es. The 
Gilroy program seems to have been much more useful as a curriculum change agent 
than as a compensatory education program. 
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Appendix 



GILROY-WLC AGREEMENT 



This \graeinent, dahed 9 July 19 70 ^ is betwean (1) Giluoy Uni 
fied School District (SCHOOL), 7663 Church Street, Gilroy, 
California 95020, and (2) Westinghouse Learning Corporation 
(WLC) a Delaware Corporation with headquarters at 100 Park 
Avenue, New York, New York 10017, 

It contains all the terms and conditions under which WLC will 
provide and the SCHOOL will purchase and use, the WLC Learn- 
ing Center Program (PROGRAM) during the 1970-71 school year. 

1, Background and purpose 

The PROGRAM has been developed by a team of psychologists. 



. ,, _ _ 

for the systematic and effective management of learning, 
valuable for remedial, regular, and enrichment purposes, com- 
pletely individualized, and selfpaced. In operation, it has 
five major elements or phases; 

— Diagnosis, The studsnt's strengths and needs are iden- 
tified through a variety of tests designed to establish 
what he already knows and what he needs to learn. 

— Prescription. A course of study is planned for each 
student, specially designed to take advantage of his 
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present aahievements and to concentrata on the areas of 
his greatest need. 

— Learning Materials. Each unit in the course of studies 
refers the student to learning materials that have been 
selected as being most effective or efficient for him 
to use in learning the content of that unit. 

— ^Motivation, Each student participates in a system for 
planning and scheduling his study programi in this way, 
he learns to assume increasing responsibility for the 
objectives and the management of his own work, of his 
study program, and this in turn motivates him to accom- 
plish it successfully and well, 

-^-"Evaluation. Progress tests measure the student's achieve 
mant in reaching his learning goals. These measures of 
achievement are used for following and aiding the stu- 
dent's progress. They are also the basis on which the 
PROGRMi is judged and paid for. 

Under this agreement ULC will establish and opc*rate a Learn- 
ing Center in Gilroy to provide reading and math instruction 
to elementary students. The objectives of the PROGBAM are 
that all students enrolled in it will (a) advance at least 
one grade level in reading and math at the end of the fiscal 
year 1971 (June 30, 1971), and (b) will further progress to 
performance levels at or near the grade level at which they 
are enrolled in school. 

2. Preparation 

A, To prepare for the opening of the Center and for the 
operation of the PROGRMI, WLC will do these things i 

(1) Not later than 20 July 1 970 , WLC will provide 
the SCHOOL with a cb^leta and detailed descrip- 
tion of the space and furnishings required to 
operate the PROGRAM so that the SCHOOL will have 
sufficient time to make suitable space ready 
for the PROGRAM prior to the beginning of the 
school year, 

(2) WLC will assign from its staff a manager to op- 
erate the PROGRAM i It is expected that the 
Center will have at least two additional staff 
members. One of these will be a teacher assigned 
to the Center from the SCHOOL staff and paid by 



the SCHOOL. WLC will also employ one or more 
aides in the Center, It is understood that the 
number of aides on duty in the Center at any 
time may be adjusted according to the number of 
students in attendance. WLC will provide all 
training required for all teachers and aides 
who will be working in the PROGRAM. 

(3) WLC will furnish all aduoatlonal equipment and 
all educational and motivational materials re- 
quired for use in the PROGRAM, (This equipment 
and these materials will remain the property 
of WLC.) 

B . To prepare for the opening of the Center and for the 
operation of the PROGRAM, the SCHOOL will do these 
things % 

(1) The SCHOOL will make available, in or near the 
Eliot School, suitable space for a Learning 
Center t accoiranodate up to 52 students. The 
space will be made ready not later than 20 
August, 1970, to meet the requirements of the 
PROGRAM as described by WLC. The SCHOOL will 
also make available adequate office space in 
or near the Learning Center for the use of the 
WLC staff manager and his saeretary. The 
SCHOOL will provide all furniture (tables, 
chairs, desks, etc.) for the Center and for the 
WLC manager's office, 

(2) The SCHOOL will select two teachers from its 
staff to work in the Learning Canter, and the 
SCHOOL agrees that WLC will have an opportunity 
to participate in and approve of their selec" 
tiun. The SCHOOL will arrange for the teachers 
selected to be available for training at least 
two weeks before the start of the school year . 

Operations 

A. WLC will operate the PROGRAM in the Center according 
to these terms and standards; 

(1) The PROGRAM will be ready to enroll students 
not later tha'^ 28 September 1970, The Center 
will be open and the PROGRAM vrill be available 



for students for no fewer than 5 hours a day, 

5 days each week during the school year. Ad- 
ditional hours of operation at any time, and 
reduced or adjusted hours of operation during 
school holiday or vacation periods will be ar- 
ranged by agreement between WLC and the SCHOOL. 

WLC will accept for enrollment in the PROGRAM 
a]l students assigned to it by the SCHOOL. 

Based on test information provided for each 
student by the SCHOOL WLC will establish a 
learning objective and a program of study for 
each student. Each student's schedule of at- 
tendance at the Center will be arranged as far 
as possible so that he may be expected to ac- 
complish his objective on schedule, 

WLC may notify the SCHOOL within the first 20 
hours of any student's attendance at the Learn- 
ing Center that in its judgment the student 
cannot benefit from the PROGRAM, and in such 
case, after review, the student will be with- 
dravrn from the PROGRAM, WLC expects that not 
more than 31 of the students will fall in this 
category. Any student who is withdrawn from the 
PROGRAM may be re-enrolled after the factors 
responsible for his withdrawal have been remedied 

The results of the PROGRAM will be measured by 
the achievement of students enrolled in it. 

The unit of achievement is one achievement- 
which is equal to a 1.0 gain in grade 
level as determined by standardized tests, 

C ® performance goal, which is subject to the 
enrollment and attendance standards established 
in parapaph 3B(2) below, is that students en- 
rolled in the PROGRAM will accomplish a total 
of 400 achievement-years. 

WLC will arrange, in cooperation with the SCHOOL, 
for visitors, observers, orientation '.leasions, 
teachers workshops , and other activities relat- 
ing to the operation of the PROGRAM provided 
only that such activities are judged not to in- 
terfere with its effective operation. 

WLC will arrange with the SCHOOL to provide it 



with appropriate information on the progress of 
each student enrolled in the PROGRAM. 

B. To assist with and support the operation of the PRO- 
GRAM, the SCHOOL will do these things: 

(1) The SCHOOL will select Title I participant stu- 
dents for enrollment in the PROGRAM during reg- 
ular school hours, based on their needs for 
remedial Instruction in mathematics and reading. 
Each student assigned will have an objective 

of achieving not less than 1.0 achievement— years 
in reading and math . 

(2) The SCHOOL will pre-test each student assigned 

to the PROGRAM in math and/or reading to establish 
his entry level. Only nationally standardized 
tests which report in grade level equivalents 
will be used for pre-testing. The SCHOOL will 
administer post-tests to each student within 
ten school days of being notified by WLC that 
the student has completed his work. The post- 
tests will be alternate lorma of the pre-tests, 
and the results of the pra- and post-tests will 
be compared to determine a student's progress 
in a subject measured in achievement-years, 

(3) The SCHOOL will be responsible for the enroll- 
ment and attendance of students in the PROGRAM 
at standard levels which will reasonably permit 
them to accomplish the PROGRAM'S performance 
goal of 400 achievement-years. To this end, 
the SCHOOL willi 

(a) Enroll students for a total of not less 
than 355 achievement-years in the Learn- 
ing Center , and 

(b) Arrange for 103 Title I students to attend 
the Learning Center for 2-1/2 hours every 
school day. This is the equivalent of 258 
student-hours per day. 

(c) Assure WLC of a "standard minimum attond- 
ance" in the Learning Center of at least 
220 student hours on not less than 170 
^^ool days during the school year. This 
means that the "standard minimum attend- 
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ance" in the Center will be 220 student- 
hours per dayj, and that the "standard 
minimum school year" will be 170 days. 

lent 



The SCHOOL will pay WLC for its success in accom- 
plishing the performance goals of the PROGEAM, and 
for the achievements of the students enrolled in it. 
The total payment to be made will be determined ac- 
cording to the following terms and conditions: 

(1) The standard price for an achievement-year ac- 
complished under this contract is 1168,75, and 
the SCHOOL will pay WLC that price for each 
achievement-year accomplished by students en- 
rolled in the PROGRAM, if the average time to 
accomplish an achievement-year in each subject 
for all students is 90 hours, or less. 

(2) If all students in the PROGRAM average more than 
90 hours per achievement-year par subject, the 
price of $168,75 will be reduced proportionately 
For example, an average of 99 hours represents 

a 10% greater time, and would result in a price 
for all achievement-years of $151.87 (90% of 
($168.75). 

(3) If any student fails to accomplish at least a 
1.0 achievement-year in a subject in 120 hours, 
the SCHOOL will pay nothing to WLC for that 
student's work in that subject. The student 
will remain in the PROGRAM, and his new pre- 
test score will be the score he obtained on his 
120-hour test. 

(4) If a student is enrolled with the objective of 
accomplishing more than a 1.0 achievement-year 
in a subject, his actual achievement, measured 
to the nearest 10th of an achievement-year, 
will be credited to the PROGRAM, and the equiva- 
lent fraction of the price for an achievement- 
year will be paid to WLC, However, the SCHOOL 
will in no ease pay for more achievement than 
was established as the student's objective when 
he enrolled, Aj.1 achievement beyond that ob- 
jective by any student will be at no cost to 
the SCHOOL, 



(5) When the SCHOOL has enrolled students for 
achievement-years having a value of $60,000 
(about 355 achievement-years) , the SCHOOL may 
elect to enroll no further students , in which 
case it will owe no further payment to WLC. 

If the SCHOOL elects to enroll students in the 
PROGRAM for more than a total value of $60,000, 

WLC will accept them for enrollment (provided 
only that there is reasonable time for them to 
accomplish the objective for which they are 
enrolled) at the price of $168,75 per achieve- 
ment-year until 400 achievement-years are ac- 
complished, and at the price of $75 per achieve- 
jnent— year for all additional enrollments to be 
completed through August 31, 1971, 

(6) If the attendance at the Learning Center on any 
of the 170 days in the "standard minimum school 
year" is less than the "standard mirimum attend- 
ance" of 220 student-hours per day, then the 
number of student— hours by which the attendance 
is less than 220 shall be considered excessive 
absence. Each hour of excessive absence will 

be considered egual to l/90th of an achievement- 
year, The total number of hours of excessive 
absences during the year, divided by 90, will 
be counted as achievement-years completed, and 
the price for that number of achievement-years 
will bo payable to WLC. Any hours of attendance 
by a student that total less than 50 in a subject, 
and all hours of attendance by a student for which 
no pr e— tes tXp^s b— tes t measurements are available 
will be considered hours of excessive absence 
for the purposes of this paragraph. WLC will 
cooperate with the SCHOOL” in schsarling additional 
hours of operation of the Learning centers to 
permit students to make up excessive absences 
and in this way to minimize the effects of this 
paragraph, 

(7) The SCHOOL will make monthly partial progress 
payments to WLC on terms to be arranged. 

5, It is understood that WLC will net be liable for loss, 
damage, detention, or delay resulting from causes beyond 
its roasonable control • 






6. MLC will use its best efforts to petform this Agreement 
in a reasonably diligent manner. There are no warranties/ 
express or implied/ eKcept as set forth m this Agreement/ 
and the results of the Learning Center system are guaranteed 
specifically as described herein and in no other way. In no 
event shall WLC be liable for any consequential or incidental 
damage arising out of this Agreement or the breach thereof. 

7 . This Agreement is not assignable by either party without 
the prior written consent of the other party. 

8. All notices given in connection with this Agreement shall 

be given in writing. If to WLC/ addressed to Westinghouse 
Learning Corporation/ 100 Park Avenue/ New York/ New York 
10017, Attention: H. K. Skeele , Vice President, and if to 

SCHOOL, addressed to Superintendent/ Gilroy Unified ■ School 
District, 7663 Church Street/ Gilroy, California 95020. 

IN WITNESS WHEREOP the parties have hereunto set their hands 
on the date first above written. 



GILROY PUBLIC SCHOOLS WESTINGHOUSE LEARNING CORPORATION 

BY: BY: 

(S, Robert Infelise) (H. K. Skeele) 



Superintendent 



Vico President 
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